System And Method For Improving An Aggregated Throughput of Simultaneous Connections

ABSTRACT

A client and content provider are connected by a plurality of simultaneous transport connections. The number of the transport connections that are used to transfer data is selected based on the size of the data to be transferred and may change after transfer of data has commenced based on the amount of data left and the attributes of the transport connections. In another aspect, data to be transmitted over the transport connections is organized into frames such that each frame includes data from only one data stream. The frames are sized to be less than or equal to a control window of the transport connection over which they are transmitted. Each frame may be assigned to a transport connection in a round robin fashion or based on the size of the frame and the sizes of the control windows of the transport connections.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/254,583, filed Sep. 1, 2016 and U.S. application Ser. No. 15/254,732,filed Sep. 1, 2016, which are hereby incorporated herein by referencesin their entirety.

This application also claims the benefit of U.S. Provisional ApplicationSer. No. 62/343,697, filed May 31, 2016, which is hereby incorporatedherein by reference in its entirety.

BACKGROUND

The amount of online content downloaded over computer networks israpidly increasing with time due to multiple factors: popularity ofonline video streaming and user-generated content, proliferation ofsocial networks and media-rich messaging, cloud-based storage etc.

Requested content often must be delivered over long distances. Only themost popular videos can be cached near the user. Content that usessecure protocols like HTTPS can't be cached without disclosing at leastsome encryption keys to third parties. Remote viewing of events streamedin real time such as sports and news can't use caching. Local content isoften not cached outside of the countries of origin.

Increase of the data travel distance often increases the rate of packetlosses. If connection uses guaranteed-delivery protocol such asTransport Control Protocol (TCP), these losses are usually interpretedas last-mile congestion, resulting in significant decrease in thethroughput even if each segment of the connection pipe is far from thesaturation point.

Spreading the data transfer across multiple simultaneousguaranteed-delivery transport connections between the sender and thereceiver usually increases the total throughput: adding each newconnection increases the aggregated size of both receiver and congestionwindows, allowing more data to be delivered during each round-tripinterval. The speed of transferring large data files over the longdistances increases with the number of simultaneous connections, untilat least one segment of the connection pipe becomes congested orresources of the server and client computers (memory, CPU load, I/Ocapacity) become overstrained.

Therefore, there is a need to increase the speed of the data transferover the multiple simultaneous connections for real-life scenarios,including bursts of limited-size data files.

SUMMARY OF THE INVENTION

In one aspect of the invention, a method for increasing an aggregatedthroughput includes establishing a plurality of simultaneous transportconnections between a first computer and a second computer over acomputer network. The method further includes receiving, by the firstcomputer from the second computer, two or more groups of requests fordata delivery to the second computer through the plurality of thesimultaneous transport connections with the first computer such that afirst group of the two or more groups of requests is constrained to befinished before a second group of the two or more groups of requests.The first computer determines whether delivery of requested datacorresponding to each request for data delivery of the first group ofrequests is finished. If so, the first computer sets a number of theplurality of simultaneous transport connections used to deliverrequested data corresponding to each request for data delivery of thesecond group of requests to the first computer such that a number ofsimultaneous transport connections used to deliver the requested datacorresponding to each request for data delivery of the first group ofrequests is set to be different than the number of the plurality ofsimultaneous transport connections used to deliver the requested datacorresponding to each request for data delivery of the second group ofrequests.

In some embodiments, each transport connection of the plurality ofsimultaneous transport connections uses a transport protocol thatincludes generating one or more acknowledgements to confirm delivery ofthe requested data corresponding to each request for data delivery ofthe first and second groups of requests. The method may further includedetermining that delivery of requested data corresponding to eachrequest for data delivery of a same group of the two or more groups ofrequests is finished in response to receiving the one or moreacknowledgements that the requested data corresponding to the eachrequest in the same group is delivered.

In some embodiments, each transport connection of the plurality ofsimultaneous transport connections uses a Transport Control Protocol(TCP).

In some embodiments, the method further includes executing, by the firstand second computers, a traffic distribution module programmed toestablish the plurality of simultaneous transport connections betweenthe first and second computers. The method may include opening, by aclient computer, one or more client transport connections to request andreceive requested data corresponding to each request of each group ofthe two or more groups of requests through the plurality of simultaneoustransport connections opened between the second computer and the firstcomputer. The client may close the client transport connections withoutinvoking closing of the plurality of simultaneous transport connectionsbetween the first and the second computer. The method may furtherinclude determining that delivery of the requested data corresponding toeach request for data delivery of a same group of the two or more groupsof requests is finished by determining that the client computer hasclosed the one or more client transport connections. The client computermay be the second computer.

In some embodiments, the second computer hosts a proxy programmed toterminate the client transport connections. In some embodiments, atleast two of the plurality of simultaneous transport connections areopened over a same physical data link. In some embodiments, at least twoof the plurality of simultaneous transport connections have identicalsource and destination network addresses.

In some embodiments, the method includes determining, by one of thefirst computer and the second computer, an estimated size of datadelivery for at least one group of the two or more groups of requests;and setting, by one of the first computer and the second computer, anumber of simultaneous network connections used to transport requesteddata corresponding to requests of a same group of requests of the two ormore groups of requests such that the number of simultaneous transportconnections used to transport the requested data corresponding to therequests of the same group of requests increases with increase in theestimated size.

In some embodiments, the estimated size is determined by obtaining oneor more network identifiers from the requests of the same group ofrequests, each identifier comprising at least one of a domain name and adestination network address.

In some embodiments, each of the plurality of simultaneous transportconnections is associated with a control window limiting the amount oftraffic that can be sent before receiving an acknowledgement that atleast some data included in the traffic has been delivered. The methodmay further include determining, by one of the first computer and thesecond computer, an estimated size of one or more control windows of theplurality of simultaneous transport connections; and decreasing, by theone of the first and the second computer, the number of the plurality ofsimultaneous network connections used for delivery of requested datacorresponding to a same group of requests of the two or more groups ofrequests with increase of the estimated size of the one or more controlwindows. In some embodiments, the control window is a smallest of areceiver window and a congestion window used by a transport protocol.

In some embodiments, the method includes changing the number of theplurality of simultaneous transport connections used to deliver therequested data corresponding to each request for data delivery of thesecond group after completing delivery of the requested datacorresponding to each request of the first group of requests and beforereceiving the second group of requests.

In some embodiments, the number of the plurality of simultaneoustransport connections used to deliver the requested data correspondingto each request of the second group of requests is decreased afterdelivery of the requested data corresponding to the first group ofrequests is followed by a period of inactivity exceeding a pre-definedtime interval before the second group of requests is submitted.

In some embodiments, the method includes changing a number of theplurality of simultaneous connections used to deliver the requested datacorresponding to a same group of requests of the two or more requests byat least one of pausing and resuming delivery of at least some of therequested data corresponding to the same group of requests through oneor more previously opened simultaneous connections, without closingexisting connections or opening new connections.

In some embodiments, the method includes setting a number of theplurality of simultaneous transport connections used for data transferfrom the first computer to the second computer to be different from anumber of the plurality of simultaneous transport connections used fordata transfer from the second computer to the first computer.

In some embodiments, changing the number of the plurality ofsimultaneous transport connections used for the data transfer from thefirst computer to the second computer while maintaining constant thenumber of the plurality of simultaneous transport connections used forthe data transfer from the second computer to the first computer.

In some embodiments, the first computer includes at least one of a proxyserver and a VPN server.

Another aspect of the invention includes a system for increasing anaggregated throughput between computers over a computer network. Thesystem includes a first computer including one or more processingdevices programmed to:

receive a group of one or more requests for data delivery to a secondcomputer through a plurality of simultaneous transport connectionsbetween the first computer and the second computer;

deliver requested data corresponding to the one or more requests fordata delivery through at least a portion of the plurality ofsimultaneous transport connections such that delivery of requested datacorresponding to each request of the one or more requests must befinished before delivery of requested data corresponding to a subsequentrequest of the one or more requests; and

-   -   change a utilized number of the plurality of simultaneous        transport connections that are used for delivery of the        requested data corresponding to the one or more requests such        that the utilized number is changed after the one or more        requests are received by the first computer but before the        delivery of the requested data corresponding to the one or more        requests.

In some embodiments, the one or more processing devices are furtherprogrammed to:

during delivery of the requested data corresponding to the one or morerequests for data delivery, generating at least one estimated valueaccording to at least one of (a) a remaining amount of data of therequested data corresponding to the one or more requests to be deliveredand (b) a parameter of data transfer through at least one transportconnection of the plurality of simultaneous transport connections usedfor the delivery;

change the utilized number in accordance with change of the least oneestimated value.

In some embodiments, the one or more processing devices are furtherprogrammed to decrease the utilized number with decrease of theremaining amount of data. In some embodiments, the one or moreprocessing devices are further programmed to estimate the remainingamount of data according to an amount of requested data corresponding tothe one or more requests that is stored in one or more buffers on thefirst computer before being sent to the second computer.

In some embodiments, the first computer includes at least one of a proxyserver and a VPN server, and the said buffers are the ones used by thatproxy or VPN server for temporary data storage.

In some embodiments, the parameter of data transfer through the at leastone transport connection is a size of a control window limiting anamount of traffic that can be sent before receiving an acknowledgementthat at least some of the traffic is delivered. The one or moreprocessing devices may be further programmed to decrease the utilizednumber with increase of the size of the control window of one or moretransport connections of the plurality of simultaneous transportconnections.

In some embodiments the parameter of data transfer through the at leastone transport connection is a rate of data losses over the at least onetransport connection. The one or more processing devices may be furtherprogrammed to increase the utilized number with increase of the rate ofdata losses over the at least one transport connection.

In some embodiments, the one or more processing devices are furtherprogrammed to change the utilized number after receiving the group ofone or more requests by one of pausing and resuming delivery of at leastsome of the requested data through the at least the portion of theplurality of simultaneous transport connections without closing any ofthe plurality of simultaneous transport connections or opening newtransport connections.

In some embodiments the utilized number is a first utilized number andthe one or more processing devices are further programmed to change thefirst utilized number independently of a second utilized number of theplurality of simultaneous transport connections that are used for datatransfer from the second computer to the first computer.

In some embodiments, the one or more processing devices are furtherprogrammed to only change the first utilized number while maintainingconstant the second utilized number.

In some embodiments, at least two transport connections of the pluralityof simultaneous transport connections are opened over a single physicaldata link. In some embodiments, the one or more processing devices arefurther programmed to change the utilized number by changing a number ofthe at least two transport connections opened over the single physicaldata link.

In one aspect of the invention, includes a method for increasing anaggregated throughput of multiple simultaneous transport connectionsbetween computers over a computer network. The method may include:

maintaining, by a first computer, a first plurality of simultaneousconnections to a second computer and a second plurality of simultaneoustransport connections to a third computer;

receiving, by the first computer, a first group of one or more requestsfor a data delivery to the second computer over the first plurality ofsimultaneous transport connections with the second computer;

delivering, by the first computer, requested data corresponding to thefirst group of one or more requests such the delivery of datacorresponding to each requests of the first group of one or morerequests must be finished before delivering data corresponding to asubsequent request to the second computer;

receiving, by the first computer, a second group of one or more requestsfor data delivery to a third computer over a second plurality ofsimultaneous transport connections with the third computer;

delivering, by the first computer, requested data corresponding to thesecond group of one or more requests such that delivery of datacorresponding to each request of the second group of one or morerequests must be finished before delivering data corresponding to asubsequent request to the third computer;

setting, by the first computer, a first number of the first plurality ofsimultaneous transport connections that are used to deliver therequested data corresponding to the first group of one or more requests;

setting, by the first computer, a second number of the second pluralityof simultaneous transport connections that are used to deliver therequested data corresponding to the second group of one or morerequests;

detecting, by the first computer, a difference between one or more firstparameters of the first plurality of simultaneous transport connectionsand one or more second parameters of the second plurality ofsimultaneous transport connections;

setting, by the first computer, the first number and the second numberto be different according to the difference between the one or morefirst parameters and the one or more second parameters.

In some embodiments, detecting the difference between the one or morefirst parameters and the one or more second parameters comprisesdetecting a difference in at least one of a distance from the firstcomputer to the second computer and a distance from the first computerto the third computer, a round-trip time between the first computer andthe second computer and a round-trip time between the first computer andthe third computer, and a quality of a network connecting the firstcomputer and the second computer and a quality of a network connectingthe first computer and the third computer.

In some embodiments, setting the first number comprises increasing thefirst number with at least one of increase in the distance between thefirst computer and the second computer and the round-trip time betweenthe first computer and the second computer. In some embodiments, settingthe second number comprises increasing the second number with at leastone of increase in the distance between the first computer and the thirdcomputer and the round-trip time between the first computer and thethird computer.

In some embodiments, setting the first number comprises decreasing thefirst number with increase of the quality of the network between theconnecting the first computer and the second computer relative to thequality of the network connecting the first computer and the thirdcomputer. In some embodiments, setting the first number comprisesdecreasing the first number with decrease in a rate of data losses overthe first plurality of simultaneous transport connections; and In someembodiments, setting the second number comprises decreasing the secondnumber with decrease in a rate of data losses over the second pluralityof simultaneous transport connections.

In some embodiments, each of the first plurality of transportconnections and the second plurality of transport connections implementa transport protocol defining control windows that limit an amount oftraffic that may be sent before an acknowledgement of at least some ofthe traffic has been received. Setting the first number may includedecreasing the first number with increase of a size of the controlwindows for the first plurality of transport connections. Setting thesecond number may include decreasing the second number with increase ofa size of the control windows for the second plurality of transportconnections.

In some embodiments, the transport protocol is a Transport ControlProtocol (TCP), and the control window is a smaller of a receiver windowand a congestion window.

BRIEF DESCRIPTION OF THE FIGURES

In order that the advantages of the invention will be readilyunderstood, a more particular description of the invention brieflydescribed above will be rendered by reference to specific embodimentsillustrated in the appended drawings. Understanding that these drawingsillustrate only typical embodiments of the invention and are nottherefore to be considered limiting of its scope, the invention will bedescribed and explained with additional specificity and detail throughuse of the accompanying drawings, in which:

FIG. 1 is a schematic block diagram of a first network environment thatmay be used to increase throughput by transferring the data throughmultiple simultaneous transport connections in accordance with anembodiment of the present invention;

FIG. 2 is a schematic block diagram of a second network environment usedto change the number of simultaneous transport connections accordinglyto embodiment of the present invention in accordance with an embodimentof the present invention;

FIG. 3 is a schematic block diagram of a third network environment usedto change the number of simultaneous transport connections in accordancewith an embodiment of the present invention;

FIGS. 4a and 4b are the graphs illustrating improvements in thethroughput that can be achieved by the embodiments of the presentinvention;

FIG. 5 is a process flow diagram of a method of changing the number ofsimultaneous transport connections in accordance with an embodiment ofthe present invention;

FIG. 6 is a diagram of a fourth network environment used to improve thetraffic distribution through simultaneous transport connections inaccordance with an embodiment of the present invention;

FIG. 7 is a process flow diagram of a method for improving the trafficdistribution through simultaneous transport connections in accordancewith an embodiment of the present invention;

FIG. 8 is a schematic block diagram of a fifth network environment usedto improve the traffic distribution through simultaneous transportconnections in accordance with an embodiment of the present invention;

FIG. 9 is a schematic block diagram of a sixth network environment usedto improve the traffic distribution through simultaneous transportconnections in accordance with an embodiment of the present invention;and

FIG. 10 is a schematic block diagram of a computer suitable forimplementing the systems and methods disclosed herein.

DETAILED DESCRIPTION

It will be readily understood that the components of the invention, asgenerally described and illustrated in the Figures herein, could bearranged and designed in a wide variety of different configurations.Thus, the following more detailed description of the embodiments of theinvention, as represented in the Figures, is not intended to limit thescope of the invention, as claimed, but is merely representative ofcertain examples of presently contemplated embodiments in accordancewith the invention. The presently described embodiments will be bestunderstood by reference to the drawings, wherein like parts aredesignated by like numerals throughout.

Embodiments in accordance with the invention may be embodied as anapparatus, method, or computer program product. Accordingly, theinvention may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.), or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “module” or“system.” Furthermore, the invention may take the form of a computerprogram product embodied in any tangible medium of expression havingcomputer-usable program code embodied in the medium.

Any combination of one or more computer-usable or computer-readablemedia may be utilized. For example, a computer-readable medium mayinclude one or more of a portable computer diskette, a hard disk, arandom access memory (RAM) device, a read-only memory (ROM) device, anerasable programmable read-only memory (EPROM or Flash memory) device, aportable compact disc read-only memory (CDROM), an optical storagedevice, and a magnetic storage device. In selected embodiments, acomputer-readable medium may comprise any non-transitory medium that cancontain, store, communicate, propagate, or transport the program for useby or in connection with the instruction execution system, apparatus, ordevice.

Computer program code for carrying out operations of the invention maybe written in any combination of one or more programming languages,including an object-oriented programming language such as Java,Smalltalk, C++, or the like and conventional procedural programminglanguages, such as the “C” programming language or similar programminglanguages, and may also use descriptive or markup languages such asHTML, XML, JSON, and the like. The program code may execute entirely ona computer system as a stand-alone software package, on a stand-alonehardware unit, partly on a remote computer spaced some distance from thecomputer, or entirely on a remote computer or server. In the latterscenario, the remote computer may be connected to the computer throughany type of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).Computer networks may use transport protocols other than InternetProtocol. Correspondingly, present invention could be implemented fortypes of network addresses other than IP addresses.

The invention is described below with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products according to embodiments of the invention. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the flowchartillustrations and/or block diagrams, can be implemented by computerprogram instructions or code. These computer program instructions may beprovided to a processor of a general purpose computer, special purposecomputer, or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via the processor ofthe computer or other programmable data processing apparatus, createmeans for implementing the functions/acts specified in the flowchartand/or block diagram block or blocks.

These computer program instructions may also be stored in anon-transitory computer-readable medium that can direct a computer orother programmable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablemedium produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide processes for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

If there is a sufficient bandwidth capacity (congestion is not anissue), the number of simultaneous connections over the long distancesis usually preset to a maximal value that can be sustained withoutoverwhelming resources of the server and client computers. However, mostof the data files transferred over a typical computer network haverelatively small sizes. Such transfers often happen in bursts, followedbut the periods of inactivity (for instance, download of a web site).Increasing the number of simultaneous connections could be lesseffective for such limited data transfers, even in the absence ofcongestion. The embodiments disclosed herein provide an improvedapproach for setting the number of simultaneous connections used totransfer data.

In one aspect of the invention, a first computer receives two or moregroups of requests for a data delivery to a second computer through aplurality of the simultaneous transport connections with the firstcomputer, where the first group of requested deliveries must be finishedbefore submitting the second group of requests. In some embodiments, thefirst computer includes means for determining that requested datadelivery is finished and means for setting the number of thesimultaneous connections used to deliver the requested data to the firstcomputer, and means for setting a different number of simultaneousconnections used to deliver the first group of requested data than thenumber of simultaneous connections used to deliver the second group ofrequested data.

In some embodiments, each of two or more simultaneous transportconnections uses a transport protocol generating acknowledgements toconfirm the data delivery, and the requested data delivery is determinedto be finished after receiving acknowledgements that requested data isdelivered. In some instances, this protocol is a Transport ControlProtocol (TCP).

In an embodiment of the present invention, the first and the secondcomputers contain traffic distribution modules enabled to establishmultiple simultaneous connections between these computers, and the datadelivery requests are generated by the client computer that opens one ormore transport connections to request and receive the data through thesimultaneous connections between the second and the first computers.According to this embodiment, the requested data delivery is determinedto be finished after the client computer closes the connections itopened to request and receive the data. In some instances, the clientcomputer is the second computer. In one more instance, the secondcomputer contains a proxy enabled to terminate transport connectionsopened by the client computer.

In some embodiments, at least two of the simultaneous transportconnections are opened over the same physical data link. In someinstances, these connections share the same source and destinationnetwork addresses.

In some embodiments, first computer obtains an estimation of the size ofat least one of the first or the second data delivery, and increases thenumber of simultaneous network connections used for correspondingdelivery with increase of the estimated size. In some instances, thesize of the data delivery is estimated by obtaining one or more networkidentifiers from the first group of requests, each identifier comprisingat least one of a domain name and a destination network address.

In another embodiment, where each of one or more simultaneously openedconnections is associated with a control window limiting the amount oftraffic that can be sent before receiving an acknowledgement that atleast some data is delivered, the first computer obtains estimated sizeof one or more control windows, and decreases the number of simultaneousnetwork connections used for the data delivery with increase of thatsize. In some instances, the control window is a smallest of a receiverwindow and a congestion window used by a transport protocol.

In some embodiments, the number of simultaneous connections used for thedata delivery is changed after finishing the delivery of the first groupof requested data, and before the second group of requests is submitted.In some instances, the number simultaneous connections used for the datadelivery is decreased after the delivery of the first requested data isfollowed by a period of inactivity exceeding the pre-defined timeinterval, before the second group of data requests is submitted.

In other embodiments, the number of simultaneous connections used forthe data delivery is changed by pausing or resuming delivery of at leastsome requested data through one or more previously opened simultaneousconnections, without closing existing connections or opening newconnections. In some instances, the number of simultaneous connectionsused for a data transfer from the first server to the second server ischanged differently from the number of simultaneous connections used forthe data transfer from the second server to the first server. In someimplementations, only the number of simultaneous connections used forthe data transfer from the first server to the second server is changed,while the number of simultaneous connections used for the data transferfrom the second server to the first server remains the same.

In some embodiments, the first computer comprises at least one of aproxy server and a VPN server.

In one aspect of the present invention, a first computer receives agroup of one or more requests for a data delivery to a second computerthrough a plurality of the simultaneous transport connections, and thedelivery of the requested data must be finished before submitting atleast one other request. First computer starts the delivery of requesteddata, and then changes the number of simultaneous connections used forthe data delivery before the delivery of the requested data is finished.

In some embodiments, the first computer obtains estimation of at leastone of the remaining amount of data to be delivered and at least oneparameter of the data transfer through at least one of the simultaneoustransport connections used for that delivery, and then changes thenumber of simultaneous connections with the change of an estimatedvalue. In some instances, first computer decreases the number ofsimultaneous connections used to deliver the remaining amount of datawith the decrease of this amount. In some instances, the remainingamount of the data to be delivered is estimated from the amount ofrequested data that are stored in one or more buffers on the firstcomputer before being sent to the second computer. In someimplementations, the first computer includes at least one of the proxyand VPN server, and said buffers are used by the proxy or VPN server fortemporary data storage.

In another instance, at least one of the parameters of the data transferthrough a transport connection is a control window, which limits theamount of traffic that can be sent before receiving an acknowledgementthat at least some data is delivered, and the number of simultaneousconnections used to deliver the remaining data is decreased withincrease of the size of one or more of these windows.

In some instances, at least one of the parameters of the data transferthrough a transport connection is an amount of data losses over thatconnection, and the number of simultaneous connections used to deliverthe remaining data is increased with the rate of data losses over one ormore transport connections.

In some embodiments, the number of simultaneous connections used for thedata delivery is changed after the group of requests is submitted bypausing or resuming data delivery through one or more previously openedsimultaneous connections, without closing existing connections oropening new connections. In some instances, the number of simultaneousconnections used for a data transfer from the first server to the secondserver is changed differently from the number of simultaneousconnections used for the data transfer from the second server to thefirst server. In some implementations, only the number of simultaneousconnections used for the data transfer from the first server to thesecond server is changed, while the number of simultaneous connectionsused for the data transfer from the second server to the first serverremains the same.

In some embodiments, at least two of the simultaneous transportconnections are opened over the same physical data link, and changingthe number of used simultaneous connections comprises changing thenumber of the simultaneous connections used for the data delivery overthat data link.

In another aspect of the present invention, the first computer receivestwo or more groups of requests for a data delivery to a second computerthrough a plurality of the simultaneous transport connections with thefirst computer, where the first group of requested deliveries must befinished before submitting the second group of requests. The firstcomputer also receives two or more groups of requests for a datadelivery to a third computer through a plurality of the simultaneoustransport connections with the first computer, where the first group ofrequested deliveries must be finished before submitting the second groupof requests. In this aspect, first computer uses the different number ofsimultaneous connections to the second computer than to the thirdcomputer, if there is a difference between one or more parameters ofconnections from the first computer to the second and the thirdcomputers.

In some embodiments, this is the difference in at least one of thedistance from the first computer, round-trip time and quality of thenetwork. In some instances, the number of simultaneously usedconnections is increased with increase of the distance to the firstcomputer or with increase of the round-trip time to the first computer.In another instance, the number of simultaneously used connections isdecreased with increase of the quality of the network between the secondand the first computer, as compared to the quality of the networkbetween the third and the first computer. In other instances, the numberof simultaneously used connections is decreased with decrease of therate of the data losses over one or more transport connections.

In some implementations, one or more transport connections use transportprotocol that sets a control window to limit the amount of data that canbe sent before receiving the acknowledgement for at least some of thesent data, and the number of simultaneously used connections isdecreased with increase of the size of one or more of the controlwindows. In some embodiments, the transport protocol is a TransportControl Protocol (TCP), and the control window is the smallest of thereceiver window and the congestion window.

In an aspect of the present invention, a system for increasing anaggregated throughput of multiple simultaneous transport connectionsbetween computers over a computer network includes a traffic managementmodule, enabled to receive one or more requests and obtain the requesteddata for a transfer from a first computer to a second computer through aplurality of simultaneous transport connections between the firstcomputer and the second computer, where the transfer of the requesteddata must be finished before receiving at least one other request fromthe second computer, a traffic distribution module, enabled to split therequested data into two or more groups and to forward two or more of thedata groups to different transport connections, and a traffic tuningmodule, enabled to obtain preferred range of the data group sizes forone or more used connections and to set the data group size for one ormore used connections within the preferred range.

In some embodiments, traffic tuning module obtains the maximal number ofavailable transport connections and sets the data group size for eachused connection, while traffic distribution module forwards each datagroup to a different transport connection, until the delivery ofrequested data is completed or until the number of used transportconnections reaches the maximal number of available connections, therebyusing smaller number of transport connections than the maximal number ofavailable connections when the size of requested delivery falls belowthe sum of preferred group sizes for available connections. In someinstances, the maximal number of available transport connections is setto the number of currently opened simultaneous connections between thefirst and the second computer, and at least one of the currently openedconnections is not used for delivery of the requested data. In someimplementations, the number of used transport connections increases withincrease of the amount of data delivered to the second computer.

In another embodiment, the traffic distribution module obtains the datareceived in response to at least two different requests from one or moretemporary buffers, forms at least one data group containing only thedata obtained in response to the same request, and then distributes thatdata group to a single transport connection. In some implementations,the data group sent to a single connection in response to a singlerequest is provided in the ordered sequence. In some embodiments, thetraffic distribution module adds headers to data groups sent to thedifferent transport connections, at least one header including anidentifier of the data corresponding to a single request and the size ofthe data group. In some instances, one or more data groups are formattedaccordingly to the Hyper Text Transport Protocol (HTTP) version 2 orhigher.

In another embodiment, each data group is forwarded to one transportconnection before the next data group is forwarded to the next transportconnection, until all requested data is forwarded to simultaneoustransport connections.

In other embodiments, the preferred range of the data group sizes is setdifferently for at least two different simultaneously used connections.

In yet another embodiment, one or more transport connections usetransport protocol setting a control window to limit the amount of datathat can be sent before receiving the acknowledgement for at least someof the sent data, and the range of preferred data group sizes increaseswith increase of the size of one or more of the control windows. In someinstances, the transport protocol is a Transport Control Protocol (TCP),and the control window is the smallest of the receiver window and thecongestion window. In another instance, the maximal value of thepreferred range for at least one connection is set to be less or equalthan the size of the control window for that connection. In otherinstances, the minimal value of the preferred range for at least oneconnection is set to be greater than the half of the size of the controlwindow for that connection.

In the other embodiment, one or more transport connections use transportprotocol limiting the amount of data that can be sent through aconnection without receiving an acknowledgement of at least somepreviously sent data, and the range of preferred data sizes is set tosend the group of data through at least one connection before receivingan acknowledgement for the previously sent data.

In other embodiments, the range of preferred data sizes is set to sendthe group of data through at least one connection in the time of oneround trip between the first and the second computers.

In another embodiment, at least two of the simultaneous transportconnections are opened over the same physical data link. In someinstances, at least two of the simultaneous transport connections havethe same source and destination network addresses.

In some embodiments, the traffic management module includes at least oneof the proxy and VPN.

In another aspect of the present invention, the system for increasing anaggregated throughput of multiple simultaneous transport connectionsbetween computers over a computer network includes at least the trafficmanagement module and the traffic distribution module. The trafficmanagement module receives two or more requests for a data transfer froma first computer to a second computer over plurality of simultaneouslyopened connections, obtains the data in response to two differentrequests, and then form two or more data frames containing response tothe different requests, each data frame containing the datacorresponding to a single request. The traffic distribution moduleforwards the data frames to two or more different transport connections,each of at least two data frames being forwarded to a single transportconnection, thereby transferring at least one data frame through eachtransport connection.

In some embodiments, traffic management module multiplexes two or moredata frames into the same queue, storing a sequence of two or more dataframes, and then traffic distribution module obtains each of at leasttwo data frames from the same queue before forwarding them to theseparate transport connections.

In the other embodiment, the number of used transport connectionsincreases with increase of the amount of requested data that isdelivered to the second computer.

In another embodiment, requested data must be delivered over thesimultaneous transport connections before receiving at least one otherrequest for a data delivery, and the sizes of one or more data framesare increased with decrease of the remaining amount of data that must bedelivered.

In other embodiments, the sizes of one or more data frames are increasedwith increased throughput of one or more transport connections betweenthe first and the second computer. In some instances, increase of thethroughput is determined by detection of at least one of the decrease ofthe packet loss rate and increase of the amount of data allowed to besent before receiving an acknowledgement for the previously sent datafrom one or more used transport connections.

FIG. 1 illustrates an example implementation of the present invention.Client computer 110 issues two or more groups of requests for datadelivery through a near proxy or VPN 120 to a far proxy or VPN 170,which passes the requests to a content provider 190 through ahigh-bandwidth pipe 180. In the presented example, near proxy or VPN 120is located at a significantly large distance from the far proxy or VPN170, increasing the rate of packet losses between the two computers. Toprevent these losses from degrading the throughput, near proxy or VPN120 may be coupled with a near multipath transport manager 130. The farproxy or VPN 170 may also be coupled with far multipath transportmanager 160, and the traffic between the near and far transport manageris distributed through multiple simultaneous connections 140, 150.

In the illustrated embodiment, simultaneous transport connections 140are used to transport the requested data; while transport connections150 remain open, but are not used to transport the requested data. Thisallows changing the number of used connections 140 to be increased ordecreased without spending the time to open new connections or to closethe old ones. In another embodiment, transport connections may be closedor opened at any time, with or without being immediately used for datatransfer.

In some implementations, at least some of the transport connections 140,150 are opened over the same physical data link covering at least partof the route between the near and far computers 110, 190. For instance,multiple connections can be opened by using the same ISP (internetservice provider), or a wireless service provider. In some embodiments,multiple connections are opened between the same source and destinationIP addresses, using only the source port to distinguish between multipleconnections. This implementation increases the probability that multipleconnections will use the same route between the client and the contentprovider, passing through the same intermediate router. In someinstances, this approach is used to extrapolate network properties of anew connection, such as a round-trip time or a packet loss rate, fromthe network properties of already opened connection, or to pre-set thecontrol window of the new connection from a control window obtained fromthe existing connection.

In another implementation, two or more simultaneous connections can beopened through different data links (for instance, both Wi-Fi andcellular connections), or can pass through the different routes due tothe differences in the destination IP addresses. This could help toselect an improved set of connections for the data transfer depending ontheir individual parameters; even if multiple connections areestablished over the same last-mile ISP to the same distant server, useof the different destination IP addresses pointing to the same servermay change the network parameters of each connection due to thedifferent routes taken by the packets between the two computers 110,190.

In some implementations, modules 120 and 170 are, correspondingly, aproxy client and a proxy server. Connections opened by the client 110are terminated by the near proxy client 120, which sends the requestpayload and receives the response payload through multiple transportconnections established between transport managers 130, 160 with the farproxy server 170. In some embodiments, transport connections 140, 150use a reliable delivery protocol such as Transport Control Protocol(TCP), which provides guaranteed delivery of the payload and benefitsfrom using simultaneous transport connections to increase aggregatedcontrol window.

In another implementation, modules 120 and 170 are, correspondingly, aVirtual Private Network (VPN) client and a VPN server. Data packetsgenerated by the client 110 are shifted to the user space by the nearVPN client 120. Near traffic manager 120 encapsulates the outgoingpackets into the simultaneous transport connections 140. Far trafficmanager 160 extracts original packets and, together with far VPN server170, sends them to the content provider 190. Data returned by thecontent provider 190 are processed in the reverse order. In this case,simultaneous transport connections help to improve throughput mostly bydecreasing the queue sizes on any intermediary routers and mitigatingbandwidth changes between the data link segments. In particular, suchintermediary routers usually maintain one queue per connection anddecreasing each queue size helps to decrease the latency and packet lossdue to overflows. If client 110 opens multiple connections using areliable delivery protocol, multiple simultaneous connections betweenVPN client 120 and VPN server 170 help to avoid decrease in performancecaused by concentrating all client connections into a single queue.

Some implementation may contain both a proxy client and a VPN client 120as well as a proxy server and a VPN server 170 used in parallel. In someembodiments, all TCP traffic originating from the client 110 isterminated by the client proxy 120 and its payload is sent throughmultiple TCP transport connections 140. In such embodiments, all UDP(user datagram protocol) traffic originating from the client 110 isencapsulated by the VPN client 120 and is sent through multiple UDPconnections.

In some implementations, near proxy or VPN 120 resides on the samecomputer as multipath transport manager 130. In some embodiments, theyreside on the same computer as the client 110. In anotherimplementation, any one or two of these components 110, 120, 130 mayreside on different computers.

In some implementations, far proxy or VPN 170 resides on the samecomputer as far multipath transport manager 160; in anotherimplementation, they may reside on different computers.

Proxy or VPN modules 120, 170 are useful when the content server 190doesn't support delivery of the same data file through the multipleparallel connections.

In an alternative embodiment where content server 190 is capable of suchsupport, the client 110 directly sends traffic to the near multipathtransport manager 130 without using a proxy or VPN client 120. In suchembodiments, the content provider 190 receives the request from andsends the data to the far multipath transport manager 160 without usinga proxy or VPN server 170. This embodiment can be realized, forinstance, if both client 110 and the content provider 190 supportMultipath TCP protocol extension.

FIG. 2 illustrates an example implementation of the near multipathtransport manager 130. In this implementation, the near multipathtransport manager 130 contains a multipath traffic distributor 220 and amultipath transport tuner 230, both connected to a near proxy or VPN120, communicating the client 110. Near multipath transport manager 130exchanges the data with a TCP stack 210, which maintains simultaneoustransport connections with the far multipath transport manager 160. Inthe depicted embodiment, multipath traffic distributor 220 receivescommands to open or close connections, or to pause or resume trafficthrough existing connections from the multipath transport tuner 230. Ifnew connection must be opened or closed, multipath traffic distributor220 sends the command to the TCP stack 210; if use of existingconnection must be paused or resumed, multipath traffic distributorchanges the distribution algorithm without the need to inform the TCPstack 210.

In the described implementation, multipath transport tuner 230 obtainsinformation from the data from a proxy or VPN 120 that helps it tochange the number of used simultaneous connections. In this example, itreceives the information about the requests issued by the client 110,including but not limited to domain (hostname), IP address, port andHTTP headers. In some embodiments, this information is used to detect apattern of requests that indicates the download of a web site of a knownsize. Size estimation is used to change the number of connections asdescribed below. Additional information may also include a content sizeprovided in response to a content request, or information from othersources such as a TCP stack 210 (which is possible only if near multipath transport manager has sufficient system privileges).

In an alternative implementation, multipath transport tuner 230 makes adecision to set or change the number of used connections 140 byobserving a round-trip time to the far multipath transport manager 160,or the time to first byte delivered after a call to the far transportmanager's 160 API.

In yet another implementations, multipath transport tuner 230 may set orchange the number of used connections based on an elapsed time since astart of a connection session, user status, or time of day, for instancewhen it correlates with known patterns of network congestion.

In addition to receiving the data needed to determine whether toincrease or decrease the number of used simultaneous connections,multipath transport tuner 230 also receives, from proxy or VPN 120,control information from the far multipath transport manager 160, suchas by receiving feedback through a persistent TCP connection to an API(application programming interface) exposed by the far multipathtransport manager 160. The multipath transport tuner 230 may also informthe far multipath transport manager 160 about the decisions to open orclose and pause or resume transport connections. Again, this may includeissuing an API call to a known network address referencing the farmultipath transport manager 160. In some embodiments, connections can beopened only by the near multipath transport manager 130. However, in atypical embodiment, both near and far multipath transport managers 130,160 can pause or resume connections.

In some implementations, near multipath transport manager uses differentconnections to send its data than the far multipath transport manager.In this implementation, number of transport connections used in onedirection is different from the number of transport connections used inanother direction; some connections are used to exchange the traffic inboth directions, while others are used only to exchange the traffic inone direction.

In another embodiment, multipath traffic distributor 220 can be combinedwith multipath transport tuner 230, or implemented as a TCP stackmodification (which usually requires root-access permissions). Insteadof communicating through the proxy or VPN 120, near multipath trafficmanager 130 can be integrated with the client 110.

FIG. 3 illustrates an example implementation of the far multipathtransport manager 160. In this implementation, the far multipathtransport manager 160 contains a multipath traffic distributor 320 and amultipath transport tuner 330, both connected to a far proxy or VPN 170,which communicates with the content provider 190. Far multipathtransport manager 160 exchanges data with the TCP stack 310, whichmaintains simultaneous transport connections with the near multipathtransport manager 130. In the depicted embodiment, multipath trafficdistributor 320 receives commands to pause and resume traffic throughalready opened connections from the multipath transport tuner 330.However, in contrast to the near multipath traffic distributor 220, themultipath traffic distributor 320 does not open or close the transportconnections in some embodiments, but rather only receives theinformation about opened or closed connections from the TCP stack 310and passes this information to the multipath transport tuner 330. Thisreflects the case when new transport connections are initiated by themodule 130 near to the client 110. If multipath transport tuner 330determines to open or close one or more connections, it passes thisinformation through the proxy or VPN 170 to the near multipath transportmanager 130.

The multipath transport tuner 330 also receives feedback about theconnections opened or closed, paused or resumed by the near transportmanager 130, and uses the information, such as a pattern of requesteddomains or IP addresses, to help it to set or change the number of usedtransport connections 140. In some embodiments, the multipath transporttuner 330 determines that delivery of requested data corresponding to agroup of requests is finished in response to determining that the client110 has invoked closing the one or more transport connections it haspreviously opened.

In some embodiments, proxy or VPN 170 is connected to a content provider190 though a pipe 180 that may have higher throughout than a pipebetween the far and the near traffic transport managers, at least forsingle connections. This could be the case, for instance, if far proxyor VPN 170 is located much closer to the content provider 190 than to anear proxy or VPN. 120. In the described embodiment, difference in thethroughput is mitigated by a receive queue 340. The receive queue 340may be embodied as a buffer storing the data downloaded from the contentprovider 190 before the data are transferred to the client 110.Multipath traffic distributor 320 reads the data from the receive queue340 before distributing it between multiple simultaneous connections140.

In the described embodiment, multipath transport tuner 330 estimates thesize of the data that remains to be delivered to the client by the sizeof the data stored in the receive queue 340. For instance, if a singlefile is loaded from the content provider 190 through the high-throughputpipe 180, and the size of the data stored in the receive queue 340starts to decrease with time, it could be interpreted as an indicatorthat this is the last remaining data for the requested delivery.Multipath transport tuner 330 may use this information to change thenumber of used simultaneous transport connections 140 after the deliverywas started.

In another instance, the client 110 may issue multiple requests todifferent content providers 190, with responses being stored in one ormore receive queues 340 before being transferred to the client 110. Inthis case, multipath traffic tuner 330 may receive information about thedata stored in each of these queues 340 and consider the data deliveryfrom the content provider 190 to be completed when each of the queues340 has less data than its maximal buffer size, and the total amount ofstored data decreases over time.

In another aspect of depicted embodiment, multipath transport tuner 330receives one or more parameters of already opened transport connectionsfrom the TCP stack 310, such as a size of a control window W (or sizesof both receiver window and a congestion window, for instance, used by aTCP transport protocol), round-trip time to a near TCP stack 210 and therate of packet losses. This information can be collected for one or moreopened transport connections. This information is usually available onthe server side because the system administrator can set higher level ofaccess privileges in a secure server environment.

In some implementations, the same far multipath transport manager 160receives connections opened by two or more near multipath transportmanagers 130 that are connected to the different clients 110. In thiscase, multipath traffic tuner 330 evaluates parameters associated withconnections to each client 110, for instance, a round-trip-time, andadvises different near transport managers 130 to open different numbersof simultaneous connections, or use different number of already openedconnections with different clients 110. After the initial number ofconnections with each client 110 is set, it can be modified before thedata delivery is completed (for instance, based on a changing size ofthe control window W or the size of the data remaining the receive queue340).

In an alternative implementation, multipath traffic distributor 320 canbe integrated with the content provider's server 190, without beingconnected to it through the far proxy or VPN 170. In this case, contentprovider's server can provide more precise information about the size ofrequested delivery, even if data reaching the multipath trafficdistributor 320 are encrypted.

In another implementation, far multipath traffic manager 160 can beintegrated with the TCP stack 310, without separately implementingmultipath traffic distributor 320 and multipath transport tuner 330.

Accordingly to the described embodiments, different number ofsimultaneous transport connections 140 can be used to deliver the datafor different group of requests, or to the different clients 110 even ifthey have requested the same content. Moreover, the number ofconnections 140 used in each case may change after the group of requestsis issued, but before the data delivery corresponding to the group ofrequests is completed.

These embodiments can be used for multiple purposes, for instance, as ameans to obfuscate proxy or VPN traffic from external censors bycreating different connection patterns for different users and differentcontent requests.

In another instance, changing the number of transport connections 140can be used to enforce fairness of bandwidth consumption over a sharedbandwidth bottleneck: the number of connections is increased only ifthere is a bandwidth capacity not utilized by the other users or if thebandwidth is a dedicated resource not shared with other users.

In some embodiments, the number of used connections 140 is changed withthe number of requested files, even if the same file is transferred overthe multiple connections. This makes it fairer to the proxy or a VPNuser, as compared with the case when multiple files are transferred overa single proxy or VPN connections.

One function of the disclosed embodiments is to tune the throughput overthe long distances between the clients 110 and content providers 190.Spreading the data transfer across multiple guaranteed-deliverytransport connections 140 between the sender and the receiver usuallyincreases the total throughput, especially when connections use atransport protocol that ensure guaranteed delivery, such as TransportControl Protocol (TCP), which limits the amount of in-flight databetween the sender and receiver to the size of the control window, whichis typically the smallest of the receiver window (amount of data thatcan be accepted by the receiver) and the congestion window (calculatedby the sender to stop a link between the sender and the receiver fromgetting overloaded with too much traffic).

In this case, adding each new connection 140 increases the aggregatedsize of both receiver and congestion windows, at least until theaggregated congestion window becomes limited by increased packet losseson the shared link. Aggregating congestion windows also has additionalbenefit of mitigating effect of the random packet losses. For instance,if the congestion window for one connection drops 50% after packet loss,the aggregated congestion window for ten parallel connections onlydecreases by 5%.

If a connection 140 is fully occupied by transporting the data, itsaverage throughput may be estimated as T=W/RTT, where W is a size of thecontrol window and RTT is an average round-trip time.

If an amount of data S is transferred over a connection 140 with controlwindow W, and assuming that this amount is sent as N packets, eachhaving a maximal segment size MSS˜=1500 B such that S=N*MSS), a time t1to deliver these data over this connection can be estimated as

t1=(N/W)*RTT   (1)

If the same amount of data is transferred over M parallel connections,each connection only has to transfer N/M packets (equivalent toaggregated connection using W*M control window), and the time tM todeliver these data becomes M times smaller:

tM=(1/M)*(N/*W)*RTT   (2)

As long as the aggregated congestion window is not limited by a sharedlink, and all parallel connections have reached a stable throughput(average W for each parallel connection is the same as for a singleconnection), a person skilled in the art may conclude that the datatransfer through M parallel connections is always faster than throughthe single connection for any M>1.

However, this conclusion implicitly assumes that the data transfercontinues after the N packets are sent through the pipe (allowingcongestion control algorithm to maintain the stable throughput). Inpractice, delivery of the requested data must often be confirmed beforeproceeding with additional data transfers. For example:

(1) A requested file must be completely downloaded and processed beforeautomatically requesting the next file (for instance, browser receives aJavaScript file, executes its code and then issues requests generated bythat code).

(2) The user issues a request for a next file or a group of files afterreviewing previously requested content (for instance, load of the website is followed by a period of inactivity until riser clicks on thelink),

(3) The amount of the data that must be confirmed before proceeding withthe next transfers usually depends on the type of requested content. Forinstance: single JavaScript file˜10 to 100 KB; news web site ˜0.5 to 5MB; video ˜=20 to 2000 MB.

Requirement to confirm delivery of N packets before proceeding with nextdata transfers changes delivery times in two ways.

First, the connection pipe may not always be filled with the maximalnumber of in-flight packets (W). For instance, if W=100 but only 10packets are sent through one connection, it will take the same time todeliver and confirm them (RTT) as for 100 packets. Increase in thenumber of parallel connections can cause the number of packets sentthrough each connection to drop below W, decreasing its throughput.

Second, the probability of losing the last packet in any connectionincreases with the number of parallel connections. If any of Mconnections loses its last packet, confirmation that all N packets aredelivered is delayed until that packet is recovered.

Most of the congestion control protocols, including the ones implementedas part of TCP, introduce an additional delay when last packet through atransport connection is lost. Congestion control protocol usually waitsfor an additional time before retransmitting such lost packet (tail lossdelay). If last packet is restored after the first retransmit, totaltime to re-deliver the last packet can be estimated astail_loss_delay+RTT=TLDF*RTT (TLDF is a time loss delay factor>1).

In some implementations of the Linux operating system, time loss delayequals to retransmit timeout (RTO): for the first retransmit,time_loss_delay˜=SRTT+4*RTTVAR, where SRTT and RTTVAR are the RTT valueand its variation, respectively, averaged over multiple ACKs. Forinstance, if RTTVAR/SRTT=0.5, TLDF˜=4.

In one known improvement implemented in the Linux operating system, thesender issues a duplicate of the last packet, known as a tail loss probe(TLP), decreasing time loss delay for the first retransmit to 2*SRTT(TLDF˜=3).

In these examples TLDF is estimated on the assumption that lost packetis re-delivered after the first retransmission attempt. If retransmittedpacket is also lost, it may increase the next RTO, or cause a switchfrom the TLP to RTO, further delaying the next retransmission attemptand increasing average TLDF.

Assuming that probability of the single packet loss (p) is the same forany packet in any connection, probability that at least one of Mconnections would lose its last packet P=1−(1−p)̂M(˜=p*M for p*M<0.5).

After accounting for incompletely filled connection pipes and tail lossdelays, times to transfer and confirm delivery of N packets through thesingle connection (t1) and through M parallel connections (tM) are:

t1=ceiling(N/W)*RTT+p*TLDF*RTT and   (3)

tM=ceiling (1/M)*(N*W))RTT(1-(1−p)̂M)*TLDF*RTT,   (4)

where ceiling(X) is a function returning closest integer K>=X.

If there are no packet losses (p=0), M can be pre-set to a high numberwithout decreasing the aggregated throughput even for small N/W: tM=RTTfor any M>N/W (delivering and confirming a single packet takes the sametime as for W packets).

However, presence of packet losses (p>0) causes aggregated throughput ofM parallel connections to decrease when M becomes larger than anestimated optimal value (M_O): spreading data transfer through multipleconnections increases the probability that at least one last packet willget lost, causing tail loss delay.

FIGS. 4a and 4b are the graphs illustrating improvements in thethroughput resulting from the present invention.

FIG. 4a depicts the change of the throughput acceleration (tM/t1) withincrease of the fill rate for each connection (N/(M*W)), computed fromequations (3,4) for N/W=10 and three combinations of p and TLDF. Itillustrates that, for realistic values of p and TLDF, best throughputacceleration is achieved when the amount of data sent through eachconnection is equal to the maximum amount that can be delivered in 1 RTT(N/(M*W)˜−1). For instance, for p=0.01 and TLDF=3, best throughputacceleration is achieved at M=10, which is up to 2 times better than forthe larger values of M.

The graph for p=0.05 and TLDF=4 shows a local minimum at N/(M*W)˜=2;with further increase of p and TLDF, it becomes an absolute minimum:when packet losses are high (e.g. network quality is low), the bestperformance can be achieved by sending maximal amount of per-connectiondata that can be delivered in 2 RTTs, slowing down the average time butdecreasing probability of the tail loss.

FIG. 4b depicts the change of throughput acceleration (tM/t1) withincrease of the fill rate for each connection (N/(M*W)), computed fromequations (3,4) for p=0.002, TLDF=3 and 3 values of N/W. It illustratesthat, for the large range of N/W, the best throughput accelerationcontinues to be achieved when N/(M*W)˜=1. If W remains the same, itmeans that, at least for the depicted range of N/W, the estimatedoptimal value of M increases proportionally with N.

If the receiver window is larger than a congestion window, W can beestimated as W=C/sqrt(p) from the Mathis equation (C˜=sqrt(3/2)). Forinstance, p=0.002 corresponds to W=27, and N/W=100 corresponds to a filesize ˜4 MB.

Equations describing the estimated optimal number of connections maydiffer from (3-4) depending on the multiple factors. For instance,variability of the W values between different connections, caused bydifferent rates of packet losses, different amounts of data previouslysent through separate connections, or by the variable round-trip timesin different connections (for instance, caused by the random retransmitsinside the last-mile wireless link).

The equations may further differ with variability of the amounts of datasent through different parallel connections caused by the datadistribution algorithms. For instance, by making the amount of data sentthrough each connection proportional to its previously observedthroughput, instead of using a round-robin distribution.

The embodiments disclosed herein do not depend on the exact form ofequations (3-4). These equations are included to validate assumptionsused by some of the embodiments of the present invention and toestablish the conditions where the embodiments could be particularlyeffective. For example, if delivery of the previously requested datamust be confirmed before proceeding with other data transfers, there isan optimal number of parallel connections that reduces the time tocomplete and confirm the delivery if packet losses are present.Likewise, the optimal number of parallel connections increases with theamount of the data required to complete and confirm the delivery. Theoptimal number of parallel connections is usually achieved when eachparallel connection is requested to deliver the maximal amount of datathat can be sent and confirmed in a single round-trip cycle, or, forespecially high rates of packet losses, in two round-trip cycles.

FIG. 5 depicts process flow diagram of a method that optimizes, e.g.,significantly improves, throughput of multiple simultaneous connectionsby monitoring parameters of the data traffic and uses these parametersto set an estimated optimal number of connections. The method of FIG. 5may be executed with respect to each near multipath transport manager130 coupled to the far multipath transport manager 160 such thatdifferent computers may use different numbers of connections 140depending on the size of the data being retrieved and attributes of theconnections between the near multipath transport manager 130 and the farmultipath transport manager 160.

In the described method, the multipath transport manager 130, 160obtains parameters of a data transfer such as a data size It in areceive queue and a size of the average control window W of existingtransport connections (step 510). Note that the size of W is determinedby TCP protocol according to factors such as round trip time, a rate ofpacket loss, a distance of a network connection between two computersimplementing TCP, or other factors describing a quality the networkconnection. The multipath transport manager 130, 160 further determineswhether this is the first data transfer after a start, a pause, a stop,a sharp decrease (e.g. 50-90% decrease) in the traffic volume for apre-defined period of time, for instance 1 s (step 520).

If a new group of requests from the client 110 is received after thestart or pause of step 520 is detected (step 540), a delivery size N isestimated from a pattern or requested domains and request headers of thenew group of requests (step 560). The number of simultaneous transportconnections is then set to optimize, e.g. significantly improve, thetime of the requested delivery corresponding to the new group ofrequests for the estimated values of R, S, and W (step 570).

While the data transfer of the requested delivery continues, at leastone of the multipath transport managers 130, 160 continues to evaluatetransport parameters such as R and W (step 520). If it detects anincrease in the number of round-trip cycles used to deliver theremainder of the data of the requested delivery, estimated asceiling(R/(M*W)) (step 530), the number of used connections 140 isincreased to decrease the time until the requested delivery is completed(step 550). Note that inasmuch as each multipath transport manager 130,160 operates independently, the number of used transport connections 140to transfer data from the client 110 to the content provider 190 may bedifferent from the number of used transport connections 140 to transferdata from the content provider to the client 110. Likewise, the numberof used transport connections 140 to transfer data from the client 110to the content provider 190 may change while the number of usedtransport connections 140 to transfer data from the content provider tothe client 110 remains constant, and vice versa.

As described above, the number of used transport connections 140 isdependent on attributes of the transport connections, including packetloss, throughput, control window, congestion window, receiver window,and other parameters that affect the value R/(M*W) (step 530). Inparticular, the size of the control window W is affected by theseattributes. Accordingly, differences in these attributes for theclient-to-content provider connections 140 and thecontent-provider-to-client connections 140 detected by the multipathtransport managers 130, 160 may result in differences in the number ofused client-to-content provider connections 14( )relative to thecontent-provider-to-client connections 140.

Instead of using the current size of the control window, oneimplementation may use the estimated size of the control window at thelater time (for instance, by assuming that the size of control windowwill continue to increase incrementally with each round trip), or byusing the Mathis equation to derive the average size of the controlwindow from the observed rate of packet losses.

Other implementations of the method that estimates one or moreparameters of the data transfer to increase the number of usedconnections may use a different sequence of steps, different ways todetect the start of a new limited data delivery or to estimate its size.

Accordingly,this and the other implementations have a common feature ofevaluating the traffic parameters after detecting a start or a pause inthe traffic, and setting the number of used simultaneous connections toa different value for different groups of requests, or for the samerequests from different users. Also, implementations that re-evaluatethe traffic parameters during the requested delivery have a commonfeature of changing the number of used simultaneous connections beforerequested delivery is completed.

FIG. 6 is a diagram of the network environment used to optimize, e.g.significantly improve, the traffic distribution of a single filetransfer through simultaneous transport connections, accordingly toanother aspect of the present invention.

In this aspect, far multipath transport manager 160 contains a set ofsend buffers 620, each send buffer is feeding a separate transportconnection 140 established by TCP stack 310. These send buffers could beimplemented in the user space, or merged with the send buffers oftransport connections provided by the TCP stack. Multipath transportmanager further contains at least one loader 630 and a receive queue640, storing the data sent through the far proxy or VPN 170 through thehigh-bandwidth 180 pipe from the content provider 190.

In the described implementation, loader 630 fills a set of send buffers620 in sequence. Before sending the data for each connection, it obtainsthe size of its control window W, extracts the amount of data thatequals W from the receive queue 640 and transfers it to the current sendbuffer, before further proceeding through the receive queue. Spaceemptied from the receive queue 640 is marked as ready to receive thenext data, for instance, by shifting the pointer in a ring buffer. FIG.6 depicts the case where the send butler for the first transportconnection has already received the amount of data equal to the size ofits control window (W1); loader 630 has obtained the size of the controlwindow for the second transport connection (W2) and is ready to transfercorresponding amount of data from the receive queue 640 to the secondsend buffer, before proceeding to the next send buffer (W3 in theillustrated example). If the amount of data left in the receive queue640 is smaller than the size of the control window (as in the exampleshown for W3), loader 630 transfers the available amount.

In some implementations, multiple loaders 630 can be used in parallel tofeed multiple send buffers 620. Likewise, different distributionalgorithms could be used instead of a round-robin (for instance,connections with the highest size of the control window could beaddressed first). The loader 630 may use parameters other than the sizeof the control window to specify amount that should be sent to each sendbuffer. For example, a value proportional to the average throughput ofthe current connection during a recent time interval (e.g. 0.1 to 10seconds).

Accordingly to the present invention, depicted implementation, and itsvariations such as described above have a common feature of sending thegroups of data to different transport connections, with the size of eachdata group being set inside a preferred range, for instance as close aspossible to the size of the control window for the correspondingconnection.

This enables automatic adjustment of the number of used connections tothe amount of the data that remains to be delivered. For instance, ifthe receive queue 640 contains only enough data to fill three sendbuffers 620 with amounts corresponding to their control window sizes W1to W3, other transport connections will not receive anymore data. Inthis way, the number of used connections 140 may raise with increase ofthe amount of data that has to be delivered, or with decrease of thesize of one or more control windows due to the random packet losses.

In some implementations, the preferred range may include only a maximalvalue, such as the size of the current control window. In otherimplementations, it may also include a minimal value. For example, ifthe receive queue 640 contains less than 50% of the size of the currentcontrol window, the loader 630 will wait for one more round-trip time toreceive the next data from the receive queue 640, in order to increasethe probability that the maximal amount of data will be sent in a singleround-trip time. Values other than 50% may be used to determine theminimal value such as a value from 40% to 60%.

If a remaining amount of data in the receive queue 640 is higher thansum of all control windows (W1 to W4 in the illustrated example), themaximal number of simultaneous connections can be set not to exceed apre-defined limit

In the depicted embodiment, data from the receive queue 640 is loadedinto each of the send buffers 620 in sequence: a first transportconnection 140 receives earlier data than a second connection 140, etc.Each data group with the size not exceeding the size of the controlwindow of the current connection 140 can be delivered during a singleround trip over the current connection. Sending a contiguous data groupin one sequence has the further benefit of reducing the amount ofreordering that has to be done on the receiving side.

In the depicted implementation, receiving side includes an orderingqueue 650, which stores data received from the multiple transportconnections 140 through the TCP stack 310, until they can be deliveredin the correct order to the proxy or VPN 170. If each connection 140delivers a contiguous set of data during each round trip, reordering isnecessary only between the groups of data (for instance, if 3^(rd)connection delivers the data faster than a second connection) or whensome data packets are lost and must be re-transmitted. If this is notimplemented, for instance if sequential data packets are spread betweenmultiple connections, packets may need to stay in the ordering queue 650until all connections deliver all data during the current cycle, therebyincreasing latency.

FIG. 7 is a process flow diagram that illustrates a method for improvingthe traffic distribution through simultaneous transport connections,based on sending the data group of the preferred size through eachconnection.

If receive queue is not empty (step 710), the next transport connectionis selected to receive a data group (step 720). In the depicted case,the next connection is selected sequentially, with a wrap over to thefirst connection when maximal number of connections is reached, :In analternative implementation, the next connection can be selected 720, forinstance, based on the amount of data remaining in its send buffer 620and the size of its control window. Specifically, connections 140 withempty send buffers 620 and large control windows would be selectedfirst. After obtaining the size of the control window for the selectedconnection (step 730), a minimum of the size of the control window (W)and a remaining amount in the receive queue (R) is moved to the sendbuffer 620 of the selected connection (step 740). The size of the dataremaining in the receive queue 640 is correspondingly adjusted to freethe space for the next data.

Inasmuch as the control window (W) is dependent on throughput and thesize of the data frames is dependent on the size of W, as outlinedabove. The size of the frames will therefore increase with increase inW. The throughput and control window are also dependent on packet lossrate. Accordingly, an increase or decrease in throughput may heestimated based on a decrease or increase, respectively, of the packetloss rate.

FIG. 8 is a diagram of the network environment used to improve thetraffic distribution of multiple parallel file transfers through thesimultaneous transport connections accordingly to embodiment of thepresent invention.

In this aspect, client 110 requests a group of files from multiplecontent providers 190. These files are downloaded in parallel throughthe proxy or VPN server 170. Before being submitted to a far multipathtransport manager 160, in the described implementation the multipledownload streams are sent through a frame assembler 860, whichmultiplexes these data streams into the single receive queue 640. Insome instances, the assembly includes creating a sequence of frames,each frame having a header and a payload corresponding to a single datastream. The header may contain an identifier of the data stream and thesize of the payload. The header may also contain additional parameters,such as the frame type (control or data, checksum, or additionaloptions). In some embodiments, each frame can be formatted according tothe specification of Hyper Text Transport Protocol (HTTP) version 2 orlater.

Accordingly to one aspect of the present invention, the loader 630transfers each successive frame from the receive queue 640 into the sendbuffer 620 of a different transport connection maintained by the TCPstack 310. In some instances, frame assembler 860 uses the same framesize for each frame. Loader 630 makes sure that each frame istransferred to the send buffer 620 of only one connection 140. In thisway, the number of used transport connections increases with the amountof data in the receive queue 640, and drops when the requested deliveryof the group of files nears the completion.

In some implementations, frame assembler 860 pre-sets the frame sizebased on the information about expected delivery size (for instance, oneobtained from a pattern for the requested domains) or the sizes ofcontrol windows used by transport connections 140 (for example, to makesure that most of the frames fit into a single control window). In someembodiments, the pre-set frame size varies for different groups ofrequests from the same user, or for different users even if they sendthe same groups of requests. Frame size may also change during the datadelivery. For instance, frame assembler 860 may decrease the frame sizeif the average size of control window decreases with time.

In some embodiments, the frame assembler 860 increases the size of oneor more data frames with decrease of the remaining amount of data to bedelivered. This approach ensures that each connection 140 transmits amaximal or near-maximal amount of data before switching to a differentconnection, where the maximal amount of data is the amount of dataallowed to be transmitted before receiving an acknowledgement of atleast some of the data. If different frames are sent through differentconnections, increasing the size of each data frame causes decrease inthe number of simultaneous connections actively used to transfer thedata. As noted above, for small files, this actually increasesthroughput when a guaranteed-delivery protocol is used.

In some embodiments the frame assembler 860 increases size of the dataframes with decrease of the packet loss rate. In other embodiments, theframe assembler 860 increases the size of the data frames with increasein the amount of data allowed to be sent before receiving anacknowledgement of at least some of the data. This approach ensures thateach connection sends the maximal amount, or near-maximal amount of databefore switching to a different connection, where the maximal amount isthe amount of data allowed to be sent before waiting for anacknowledgement of at least some of the data. Decrease of the data lossrate, or increase of the amount of data allowed to be sent beforereceiving an acknowledgement (increase of the control window) results indecreasing the number of simultaneous connections that are usedaccording to the method of FIG. 5. Increasing the size of each dataframe, achieves a similar result: different frames are sent throughdifferent connections such that increasing the size of each data framecauses a corresponding decrease in the number of simultaneousconnections 140 actively used to transfer the data, particularly forsmall files.

In some instances, loader 630 selects the next connection 140 for a nextframe in the receive queue 640 sequentially. In another instance, it mayselect the next connection 140 based on the size of its control windowin order to increase the number of connections that transfer a singleframe in the time of a single round trip. This becomes beneficial whenframe assembler 860 forms frames of different sizes, for instance byusing smaller frames for high-priority data (such as JavaScript filesthat may stall web site rendering) to speed up their delivery.

Accordingly to the illustrated implementation and its variationsdescribed above have a common feature of ordering the data transfer ofthe multiple data streams through multiple transport connections 140 byforming data frames, each frame containing only the data from a singledata stream, and sending each data frame through only one transportconnections; 140. Data frames from the same data stream can be sentthrough the multiple simultaneous transport connections 140, but eachdata frame can be sent only through a single transport connection.

This implementation, in addition to automatically adjusting the numberof used connections, also decreases the need for data reordering on thereceiver side. In the depicted embodiment, the receiver side includesthe ordering queue 650. It sends the sequence of frames to the framedisassembler 870, which de-multiplexes them into the separate datastreams sent through the proxy or VPN 170 to the multiple contentservers 190. Ordering queue 650 only has to make sure that each framecontains all its data. As soon as all packets of the frame are deliveredby a single transport connection, that frame can be immediately sent tothe frame disassembler 870. In this case, ordering queue 650 never hasto wait for two or more transport connections 140 before delivering anindividual data frame.

FIG. 9 is a diagram of the network environment used to optimize, e.g.significantly improve, the traffic distribution of multiple parallelfile transfers through simultaneous transport connections accordingly toanother embodiment of the present invention.

In this embodiment, multipath transport manager 160 contains multiplereceive queues 640, each used to store data for a single data streamloaded from a corresponding content provider 190 through the proxy orVPN 170. In this embodiment, loader 630 also performs a function of aframe assembler. Specifically, the loader 630 selects an amount of datathat has to be taken from the next receive queue 640, formats it as aframe and loads the frame into the next send buffer 620, which is usedby the TCP stack 310 to transfer the data through the correspondingtransport connection 140. The loader 630 then performs this functionwith respect to another receive queue 640 and transport connection 140.

In the described embodiment, frame assembler and loader 630 obtains thesize of the control window W for each transport connection 14o and usesit to optimize the frame size, e.g. select a frame size providingimproved results. For example, each frame size is set as close aspossible to the size of the corresponding control window, maximizing, orat least increasing, the probability that the frame will be delivered inthe time of a single round trip. In the depicted embodiment, Frame 1 ofsize W1 is already loaded from the first receive queue 640 to the firstsend buffer 620. The available space of the first receive queue 640 isthen adjusted, for instance, by changing the pointer of a ring buffer.Frame assembler and loader 630 is then ready to assemble a frame of thesize W2 from the second receive queue 640, and then to advance to thethird receive queue 640. If the number of transport connections 140 islarger than the number of receive queues 640, queues 640 may startrotating from the beginning. In the illustrated example, the fourth sendbuffer 620 will receive the remaining data from the first receive queue640, etc.

In another implementation, the loader 630 sets the frame size for dataloaded into a send buffer 620 based on a previously measured throughputof the transport connection 140 corresponding to the send buffer 620. Inanother example, the loader 630 sets the frame size based on the type ofconnection 140 (for instance, wired versus wireless).

In another implementation, a next transport connection 140 may beselected by the loader 630 to receive a next frame using an algorithmthat is different from a round-robin algorithm. For instance, byselecting, as the next connection 140, the connection 140 that has thelargest size of control window or largest throughput over a recent timeinterval (e.g. 0.1 to 10 seconds) of those connections 140 that have anempty send buffer 620. The next receive queue 640 can also be selectedby using a different algorithm. For instance, different receive queues640 may be assigned different priorities and data may be assigned to thequeues such that higher priority data will he assigned to a higherpriority queue 640. For example, JavaScript files that may stall website load could be assigned to a higher priority queue 640. In anotherexample, the queue 640 with smallest amount of remaining data may beassigned a higher priority in order to speed up the download of smallfiles in the presence of large background downloads.

The depicted implementation and its variations as described above have acommon feature of ordering the data transfer of multiple data streamsthrough multiple transport connections 140 by forming data frames, eachframe containing only the data from a single data stream. Each dataframe is sent through only one transport connection 140, and its size isselected for that connection to increase the throughput (for instance,by improving the probability that each frame will be sent out during asingle round trip) relative to a range of other possible sizes.

Sending data frames of different sizes through the different connectionsalso provides a benefit of a simplified reordering on the receiver side.In the described implementation, data from each transport connection 140are stored in a separate reordering queue 650. In the depicted example,only three transport connections carry the upstream data, while anotherone remains unused. Each data frame may only be sent through a singletransport connection 140, but each transport connection 140 may carrydata frames from more than one data stream.

The depicted embodiment may benefit from the fact that data frames sentthrough the same transport connection 140 have the same or a similarsize, which makes easier to anticipate how much memory would be requiredfor a connection-specific re-ordering buffer, thereby increasingefficiency of memory allocation. It also makes it lessresource-consuming to re-order the frames from multiple connections inparallel. As soon as all packets from any frame in any ordering queueare delivered, they are sent to the frame disassembler 870, whichdemultiplexes them into the separate data streams and forwards each datastream through the proxy or VPN 170 to the corresponding contentprovider 190.

Depicted embodiments reference Transport Control Protocol (TCP) and aTCP stack 310 as a means for the guaranteed data delivery over themultiple simultaneous connections. Other embodiments may use differentdelivery protocols, for instance, a User Datagram Protocol (UDP) withadditional user-space flow control acid congestion control modules, or aStream Control Transmission Protocol (SCRIP).

In the other embodiments, multipath traffic managers 130, 160 may beintegrated with a client application executing on the client 110 andcontent provider's servers 190, respectively, thereby eliminating theneed for the separate proxy or VPN modules 120, 170. For instance, theembodiments disclosed herein can be used to accelerate throughput of theMultipath TCP protocol extension, if it is supported by both the client110 and the content provider 190.

In some embodiments, different data frames are sent through thedifferent transport connections 140 sharing the same physical data linkor all data links on the same route. In the other embodiments, differenttransport connections may use different data links, such as wired andwireless, or different routes to the same server having multiple IPaddresses.

Multiple simultaneous connections may be used to transfer all trafficfrom the client 110 to the proxy or VPN server 120 or only part of thetraffic. In some embodiments, only long-distance content is accessedthrough the multiple simultaneous connections 140 to mitigate the effectof the random data losses whereas content near the user is accesseddirectly, without spreading the load of the same file across multipleconnections 140.

FIG. 10 is a block diagram illustrating an example computing device 1000which may embody any of the computers and servers disclosed herein.Computing device 1000 may be used to perform various procedures, such asthose discussed herein. Computing device 1000 can function as a server,a client, or any other computing entity. Computing device can performvarious monitoring functions as discussed herein, and can execute one ormore application programs, such as the application programs describedherein. Computing device 1000 can be any of a wide variety of computingdevices, such as a desktop computer, a notebook computer, a servercomputer, a handheld computer, tablet computer and the like.

Computing device 1000 includes one or more processor(s) 1002, one ormore memory device(s) 1004, one or more interface(s) 1006, one or moremass storage device(s) 1008, one or more Input/Output (I/O) device(s)1010, and a display device 1030 all of which are coupled to a bus 1012.Processor(s) 1002 include one or more processors or controllers thatexecute instructions stored in memory device(s) 1004 and/or mass storagedevice(s) 1008. Processor(s) 1002 may also include various types ofcomputer-readable media, such as cache memory.

Memory device(s) 1004 include various computer-readable media, such asvolatile memory (e.g., random access memory (RAM) 1014) and/ornonvolatile memory (e.g., read-only memory (ROM) 1016). Memory device(s)1004 may also include rewritable ROM, such as Flash memory.

Mass storage device(s) 1008 include various computer readable media,such as magnetic tapes, magnetic disks, optical disks, solid-statememory (e.g., Flash memory), and so forth. As shown in FIG. 10, aparticular mass storage device is a hard disk drive 1024. Various drivesmay also be included in mass storage device(s) 1008 to enable readingfrom and/or writing to the various computer readable media. Mass storagedevice(s) 1008 include removable media 1026 and/or non-removable media.

I/O device(s) 1010 include various devices that allow data and/or otherinformation to be input to or retrieved from computing device 1000.Example I/O device(s) 1010 include cursor control devices, keyboards,keypads, microphones, monitors or other display devices, speakers,printers, network interface cards, modems, lenses, CCDs or other imagecapture devices, and the like.

Display device 1030 includes any type of device capable of displayinginformation to one or more users of computing device 1000. Examples ofdisplay device 1030 include a monitor, display terminal, videoprojection device, and the like.

Interface(s) 1006 include various interfaces that allow computing device1000 to interact with other systems, devices, or computing environments.Example interface(s) 1006 include any number of different networkinterfaces 1020, such as interfaces to local area networks (LANs), widearea networks (WANs), wireless networks, and the Internet. Otherinterface(s) include user interface 1018 and peripheral device interface1022. The interface(s) 1006 may also include one or more user interfaceelements 1018. The interface(s) 1006 may also include one or moreperipheral interfaces such as interfaces for printers, pointing devices(mice, track pad, etc.), keyboards, and the like.

Bus 1012 allows processor(s) 1002, memory device(s) 1004, interface(s)1006, mass storage device(s) 1008, and I/O device(s) 1010 to communicatewith one another, as well as other devices or components coupled to bus1012. Bus 1012 represents one or more of several types of busstructures, such as a system bus, PCI bus, IEEE 1394 bus, USB bus, andso forth.

For purposes of illustration, programs and other executable programcomponents are shown herein as discrete blocks, although it isunderstood that such programs and components may reside at various timesin different storage components of computing device 1000, and areexecuted by processor(s) 1002. Alternatively, the systems and proceduresdescribed herein can be implemented in hardware, or a combination ofhardware, software, and/or firmware. For example, one or moreapplication specific integrated circuits (ASICs) can be programmed tocarry out one or more of the systems and procedures described herein.

Reference throughout this specification to “one embodiment,” “anembodiment,” “one example,” or “an example” means that a particularfeature, structure, or characteristic described in connection with theembodiment or example is included in at least one embodiment of thepresent disclosure. Thus, appearances of the phrases “in oneembodiment,” “in an embodiment,” “one example,” or “an example” invarious places throughout this specification are not necessarily allreferring to the same embodiment or example. Furthermore, the particularfeatures, structures, or characteristics may be combined in any suitablecombinations and/or sub-combinations in one or more embodiments orexamples. In addition, it should be appreciated that the figuresprovided herewith are for explanation purposes to persons ordinarilyskilled in the art and that the drawings are not necessarily drawn toscale.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrative,and not restrictive. The scope of the invention is, therefore, indicatedby the appended claims, rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed is:
 1. A system for increasing an aggregated throughputbetween computers over a computer network, the system comprising a firstcomputer including a processing device including one or more processors,one or more memory devices coupled to the one or more processors andencoded with instructions that when executed by the one or moreprocessors cause the first computer to: receive a group of one or morerequests for data delivery to a second computer through a plurality ofsimultaneous transport connections between the first computer and thesecond computer; deliver requested data corresponding to the one or morerequests for data delivery through at least a portion of the pluralityof simultaneous transport connections such that delivery of requesteddata corresponding to each request of the one or more requests must befinished before delivery of requested data corresponding to a subsequentrequest of the one or more requests; and change a utilized number of theplurality of simultaneous transport connections that are used fordelivery of the requested data corresponding to the one or more requestssuch that the utilized number is changed after the one or more requestsare received by the first computer but before the delivery of therequested data corresponding to the one or more requests.
 2. The systemof claim 1, wherein the one or more memory devices are further encodedwith instructions that when executed by the one or more processorsfurther cause the first computer to: during delivery of the requesteddata corresponding to the one or more requests for data delivery,generating at least one estimated value according to at least one of (a)a remaining amount of data of the requested data corresponding to theone or more requests to be delivered or (b) a parameter of data transferthrough at least one transport connection of the plurality ofsimultaneous transport connections used for the delivery; change theutilized number in accordance with change of the least one estimatedvalue.
 3. The system of claim 2, wherein the one or more memory devicesis further encoded with instructions that when executed by the one ormore processors further cause the first computer to decrease theutilized number with decrease of the remaining amount of data.
 4. Thesystem of claim 3, wherein the one or more memory devices are furtherencoded with instructions that when executed by the one or moreprocessors further cause the first computer to estimate the remainingamount of data according to an amount of requested data corresponding tothe one or more requests that is stored in one or more buffers on thefirst computer before being sent to the second computer.
 5. The systemof claim 4, wherein the first computer includes at least one of a proxyserver or a VPN server, and the said buffers are the ones used by thatproxy or VPN server for temporary data storage.
 6. The system of claim2, wherein the parameter of data transfer through the at least onetransport connection is a size of a control window limiting an amount oftraffic that can be sent before wherein the one or more memory devicesare further encoded with instructions that when executed by the one ormore processors further cause the first computer to decrease theutilized number with increase of the size of the control window of oneor more transport connections of the plurality of simultaneous transportconnections.
 7. The system of claim 2, wherein the parameter of datatransfer through the at least one transport connection is a rate of datalosses over the at least one transport connection; and wherein the oneor more memory devices are further encoded with instructions that whenexecuted by the one or more processors further cause the first computerto increase the utilized number with increase of the rate of data lossesover the at least one transport connection.
 8. The system of claim 1,wherein the one or more memory devices are further encoded withinstructions that when executed by the one or more processors furthercause the first computer to change the utilized number after receivingthe group of one or more requests by one of pausing and resumingdelivery of at least some of the requested data through the at least theportion of the plurality of simultaneous transport connections withoutclosing any of the plurality of simultaneous transport connections oropening new transport connections.
 9. The system of claim 8, wherein theutilized number is a first utilized number; wherein the one or morememory devices are further encoded with instructions that when executedby the one or more processors further cause the first computer to changethe first utilized number independently of a second utilized number ofthe plurality of simultaneous transport connections that are used fordata transfer from the second computer to the first computer.
 10. Thesystem of claim 9, wherein the one or more memory devices are furtherencoded with instructions that when executed by the one or moreprocessors further cause the first computer to only change the firstutilized number while maintaining constant the second utilized number.11. The system of claim 1, wherein at least two transport connections ofthe plurality of simultaneous transport connections are opened over asingle physical data link; and wherein the one or more memory devicesare further encoded with instructions that when executed by the one ormore processors further cause the first computer to change the utilizednumber by changing a number of the at least two transport connectionsopened over the single physical data link.